NOTICE 


THIS DOCUMENT HAS BEEN REPRODUCED FROM 
MICROFICHE. ALTHOUGH IT IS RECOGNIZED THAT 
CERTAIN PORTIONS ARE ILLEGIBLE, IT IS BEING RELEASED 
IN THE INTEREST OF MAKING AVAILABLE AS MUCH 
INFORMATION AS POSSIBLE 



AgRISTARS 


iwlliW® uiHJ®r NASA spoWOtlW^ 
in the interest of early m<1 wW# rll*’ 
semlnaVw-n ol earth 

Program iotor%ation and without llibillty 

for afiif w? ^ii0& thereof^ 


Supporting Re^arch 


SR-PO-0046fiO- lOtlgO 

NAS9-15466'^- ' 

/£,}>i7l 

A Joint Program for 
Agriculture and 
Resources Inventory 
Surveys Through 
Aerospace 
Remote Sensing 

March 1980 



Technical Report 

h 

\! 

On the Accuracy of 

Pixel Relaxation Labeling I 

by J.A. Richards, D.A. Landgrebe, 
and PH. Swain 

(E 80 - 10185 ) ON THE ACCUBACY CE BIXEI N 80-27705 I 

RELAXATION LABELING (Burdue Univ.) 34 p I 

HC A 03 /HE A 01 CSCL 05 B | 

Unci as i 

G 3/43 00185 

Laboratory for Applications of Remote Sensing 
Purdue University 



star Information Form 


1. Ror/ort No 2. Government Accession No, 

SR- PO- 004 5 5 

3. Recipient’s Catalog No. 

4 Title and Subtitle 

On the Accuracy of Pixel Relaxation Labeling 

5 Report Date 

March 1980 

6. Performing Organiirallon Code 

7. Aulhor(s) 

J. A. Richards, D. A. Landgrebe, P. H. Swain 

8. Performing Organisation Report No. 

030180 

Laboratory for Applications of Remote Sensing 

Purdue University 

1220 Potter Drive 

West Lafayette, IN 47906 

Id. Work Unit No. 

11, Contract or Grant No 

NAS9-1 5466 

13. Type of Report and Period Covered 

t2. Sponsoring Agency Name and Address 

J. D. Erickson 

NASA/ Johnson Space Center 

Hrtlilfnn TV 77nRft - . 

14 Sponsoring Agency Code 

15 Supplemonlary Notes 

16. Abstract 


An analysis of pixel labeling by probabil istic relaxation techniques is 
presented to demonstrate that these labeling procedures degenerate to weighted 
averages in the vicinity of fixed points. A consequence of this is that 
undesired label conversions can occur, leading to a deterioration of labeling 
accuracy at a stage after an improvement has already been achieved. Means for 
overcoming the accuracy deterioration are suggested and are used as the basis 
for a possible design strategy for using probabilistic relaxation procedures. 

The results obtained are illustrated using simple data sets in which 
labeling on individual pixels can be examined and also using Landsat imagery 
to show application to data typical of that encountered in remote sensing 
appl i cations . 


17. Key Words (Suggested by Author(s)) ! 

Pixel Labeling 

18. Distribution Stalomenl 



19. Security Classlf. (of this report) 

u 

20. Security Classlf. (of this page) 

u 

21. No. of Pages 

31 

22. Price’ 


•For sale by tho National Technical Intormallon Service, Springfield, Virginia 22161 NASA - JSC 








SR-PO-00455 
NAS9-1 5466 


TECHNICAL REPORT 

ON THE ACCURACY OF PIXEL RELAXATION LABELING 


By 


J. A, Richards 
D. A. Landgrebe 
P. H. Swain 


This report describes activity carried 
out in the Supporting Research Project. 


Purdue University 

Laboratory for Applications of Remote Sensing 
West Lafayette, Indiana 47907, U.S.A. 


MARCH 1980 


I 


LARS Technical Report 030180 


"ON THE ACCURACY OF PIXEL RELAXATION LABELING"* 

J. A. Richards 
D. A, Landgrebe 
P, H. Swain** 


ABSTRACT 

An analysis of pixel labeling by probabilistic relaxation 
techniques is presented to demonstrate that these labeling procedures 
degenerate to weighted averages in the vicinity of fixed points. A 
consequence of this is that undesired label conversions can occur, 
leading to a deterioration of labeling accuracy at a stage after an 
improvement has already been achieved. Means for overcoming the 
accuracy deterioration are suggested and are used as the basis for 
a possible design strategy for using probabilistic relaxation procedures. 

The results obtained are illustrated using simple data sets in 
which labeling on individual pixels can be examined and also using 
Landsat imagery to show application to data typical of that encountered 
in remote sensing applications. 
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I. INTRODUCTION 

Probabilistic relaxation procedures, which employ the information 
embedded in spatial context, appear to be attractive techniques for 
reducing labeling errors in various types of image data. In the results 
of some simple exercises such as the labeling of the sides of a triangle 
[1,2], this has indeed been the case with perfect labeling shown to be 
possible. However, in more complex labeling tasks such as line and 
edge enhancement [3,4,5] and pixel labeling [6,7], the results obtained 
to date detract somewhat from the appeal of relaxation since labeling 
accuracy has been observed to improve during the early iterations of the 
process only to be followed by a subsequent degradation. In pixel labeling, 
for example, the labeling error exhibits a turning point at a specific 
iteration and the final error, in some situations, can be worse than that 
initially; similarly in line enhancement applications, line broadening 
is observed to occur late in the process, degrading an otherwise acceptable 
labeling. From a practical viewpoint, this suggests that the relaxation 
process in these sorts of applications should be stopped at some particular 
point to avoid incipient deterioration of the results. However, since the 
iteration of minimum error will not be known, so that the optimum stopping 
point will not be evident, it is likely that the final labeling error will 
always be larger than necessary. To avoid 'i;his situation, it is clearly 
important that the degradation mechanism be understood so that, at worst, 
a stopping criterion can be devised or, better still, the deterioration 
of labeling accuracy can be minimized or avoided, Eklundh and Rosenfeld 
[8] and Peleg [9] have addressed the task of detenr.ining suitable stopping 
rules. In particular, Eklundh and Rosenfeld observe that the convergence 
of relaxation is such that labeling error changes most in the earlier 


I-erations and only slightly in the 'latter stages of the process. As 
a result, they recommend that the average absolute difference between label 
estimates in pairs of sequential iterations be computed and .that relaxation 
be terminated when this measure is an order of magnitude smaller than it 
was after the first iteration. Peleg bases his stopping criterion on a 
measure of the likelihood that the labeling at each iteration is the 
correct one. By establishing a likelihood measure that incorporates both 
the influence of the current label estimates and the effect of the current 
joint probabilities, he demonstratts (for two examples) that the most 
probable labeling occurs at some iteration before that where the minimum 
labeling error is observed. A second stopping rule proposed by Peleg, 
using information channel concepts, exhibits similar behavior. With both 
of these measures, the process is stopped significantly short of the 
iteration of minimum error and thus they must be regarded as sub-optimal 
criteria. Moreover, since the reason for the turning point in the error 
curve has not been determined, there is no theoretical reason to suppose 
that stopping rules of these types will circumvent deterioration of labeling 
accuracy. 

The present treatment is directed towards understanding the mechanism 
during relaxation that causes labeling error to increase again after having 
reached a minimum. It is demonstrated that this is a process of local 
averaging once relaxation has approached a fixed point. As a consequence, 
it is shown that if the parameters in the relaxation algorithm are suitably 
chosen, the error versus number of iterations curve can be made to decrease 
monotonical ly to a fixed error. 
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2- THE RELAXATION ALGORITHM AND THE DEFINHION OF NEIGHBORHOOD 

Consider the probabilistic relaxation algorithm of Rosenfeld, Hummel 
and Zucker [1]: 

P^-'^(A)Qi'^(A) (1) 

' X 

where p^ (A) is the k estimate of the probability that A is the proper 
label for the i^*^ pixel, and Q^.*^(A) is the k^^ estimate of the neighborhood 
function, given by 

Q.''(X) = 1 + L d, 

’ jeJ ^ 

In this expression the r. j(aIa') are the compatibility coefficients, the 

' V 

d^ are a set of neighbor weights that can be used to give different 
neighbors differing degrees of influence in the neighborhood function, 
and J defines the neighborhood about the particular pixel being considered. 
Owing to practical considerations, this neighborhood in pixel labeling is 
chosen either as the 3x3 set of pixels about the pixel under consirieration 
or, more simply, as the pixels above, below, and to the sides of that 
pixel. Within these choices two variations appear to have been used. 

One includes the central pixel (i.e., that under consideration) as a 
member of that neighborhood [1] and the other excludes that pixel [10,11]. 
These will be referred to here as inclusive and exclusive neighborhoods 
respectively. The following analysis is based upon inclusive neighborhoods 
with results for exclusive neighborhoods, as required, being given as 
special cases. 

3 • LOCAL /\V ERA CiIMR IN THE VICINITY OF FIXED POINTS AND ITS EFFECT ON 
GEOMETRIC FEATURES 


L n.(AlA')p*J(A'). (2) 

A' J 


Suppose a particular relaxation exercise has progressed to a point 
where the label estimates have all approached 0 or 1 . (The stage where 
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the label estimates are at 0 or 1 is called a fixed point in the process. 

I, 

Fixed points with p^ (X) other than 0 or 1 can occur*, however, they are 
infrequent in pixel labeling and will not be considered here.) Within 
homogeneous regions — i.e., where all pixels in a neighborhood have the 
same predominant label — the mutual support offered among neighbors will 
not allow the label estimate on any particular pixel to alter by any 
significant amount. In fact, those estimates will simply move closer to 
their fixed points. However, the situation at boundaries such as corners 
of one region within another, the ends of lines (single pixel wide), and 
isolated pixels can be quite different, as the following discussion 
reveals. 

Consider a Xi pixel on the boundary between Xi and regions. 

Evidently p^.*^(Xi) is the largest label estimate for that pixel and it is 

reasonable to assume for such a Xi,Xa neighborhood that p^.*^(Xji) > p^.**'(A?) » 

k k 

p,- (X„), Vn f 1,2. Now consider whether the label estimate p,- (Xi) will be 
i n 1 

strengthened or weakened as relaxation proceeds. To do this, it is sufficient 
to consider the relative strengths of the neighborhood functions as defined 
in (2). In particular, if 

Q^.'^(Xi) > Q.'^(X?J 

the Xj label will be strengthened at the next iteration; otherwise it will 
weaken. This wiP continue with subsequent iterations (since the label 
estimates at neighbors will not change by any significant amount). Should 
Qj (X?) > (Xi), the repeated application of relaxation will ultimately 

lead to being the favored label at the pixel -- i.e., the Xi label will 
be removed by further iterations. Consequently even though labeling error 
could have been reduced in establishing the Xi label on that pixel, it 


0 - 


win now (gradually) increase owing to the loss of that label. To avoid 
this, it is necessary, therefore, to ensure (from (2)) that 

dj L r.j(X2lX')Pj‘^(X') 



1 .e. 


dj L {r^j(XilX') - r^.j(X2lX')} Pj'^(X') > 0 
jcJ X* 


(3) 


Note that the additive '*!" in (2) has been of no significance in determining 
(3), so that (3) is a result general to all present relaxation algorithms 
which employ arithmetic averaging over the neighborhood, including 
particularly that where the r--(X|X') are mapped to conditional probabilities 

' J 

in which case the "1" does not appear in (2). (See [10].) 

The probability that the pixel's label could alter to that of a third 
class X 3 has been ignored owing to the earlier assumptions regarding the 
relative strengths of the label estimates on that pixel. 

Since it has been assumed that all the probability estimates are 
close to 0 or 1, (3) can be modified to 


I 


''j 


{i-i.(x.Uj) 


>0 


(4) 


where X. is the preferred label on the j*” neighbor. 

u 

Now consider the neighborhood definition explicity. Let O' be 
the exclusive neighborhood so that 0 : {J',i} where i is the pixel whose 
label is "currently" under consideration. Then (4) can be recast as 
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dj (^1 !^i) " |Ai)} + dj{r^ j(Xi |Xj) - >'^j(X?|Xj)) J* 0, 

L - '•ij(^'i^j)) (5) 

giving d^. > — — 

r^^ihlU) - r^,(XjlAi) 

as the condition that Xi be retained as the label for the 1^*^ pixel. 

To simplify further discussion, now consider some special cases of 
(5), First suppose the compatibilities r. .(X|X') have been chosen as 
conditional probabilities, and secondly consider only a two-label problem 
so that 

Thus (5) becomes 

E d,n - 2p,.(x,|x,)} 

ieJ' d IJ 0 

di > - — — 

-1 + 2p..(XilXi) 

Within the conditional probability compatibility definition, it is 
logical that p^-^-(XijXi) - 1, although in (5) r^-^(Xi|Xi) based upon other 
compatibility definitions need not necessarily be unity. To avoid loss 
of a Xi label on a border between Xi and Xz regions, we therefore have 
the condition 

“1 " Jj, '•j'' - (6) 

Now consider the particular choice of neighborhood shown in Fig. I, and 
leu the pixel under consideration be a corner pixel, as depicted. Suppose 
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it 

are the same for each neighbor j of the corner pixel, In view of these, 
(6) simplifies to 

d| > 4d {p^jCXaUa) - P^j(XilXi) 

Further suppose the dj have been chosen such that Jclj » 1. Such 
a choice is strictly only required when the r^{X [ X') are chosen as 
correlations. However, it is a useful choice general and here leads 
to 4d + d^. * 1 so that we have 

d^. > n(l+n)”\ n * p^j(X 2 lX 2 )-p^j(Xi|Xi) (7) 

as the required condition that Xi corner labels not be lost, This 
condition also applies to the preservation of single-pixel -wide Xj 
lines that pass through a X? neighborhood. For the simple neighborhood 
chosen, the only other ^/lometries that are subject to label conversion 
(deterioration) by the mechanism described are the ends of lines a single 
pixel wide, and single isolated pixels. From (6) it can be shown that the 
condition for the preservation of labels at the ends of lines of Xi 
within Xy regions is 


dj « d Vj, and further assume the compatibility coefficients p^^{X 


d. > 


3p..(X2lX2) - p^.(Xi!Xi) - 1 


(8) 


3p^.j(X2lX2) - p.j(XilXi) + 1 
Likewise, to preserve individual Xi labeled pixels in Xz regions, it is 
necessary that 


2p^.j(X?, jXg) - 1 

d. > — ' — ~ 

^ 2p.j.(XzlX2) 


(9) 


*This ignores, for example, systematic biases such as the unequal vertical 
and horizontal sampling rates present in Landsat imagery. 
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ihe predictions of (7-9) were checked using the data chosen in 1 

i 

Figure 2. This is assumed to be a portion of an image for which the i 

compatibilities are p^.j(WlW) » 0,700,p^j(b|b) == O.BOO, where b implies ] 

blank. Using (7-9), the following conditions can be determined. 

1. To avoid loss of a W corner in a b region d^ > 0,091 

2, To avoid loss of a b corner in a W region d^.>- 0.111 

3, To avoid loss of a W line end in a b H-H|1on d^ > 0.259 

4. To avoid loss of a b line end in a W region d^. ?■ 0,130 i 

5. To avoid loss of a W pixel in a ^b region d^. > 0.375 ' 

6 , To avoid loss of a b pixel in a W region d. 0.286 i 

1 j 

Consequently we would expect that if I 

! 

0, only b corners would be retained | 

d| 0.100 both b and W corners would be retained 

d^. 0,150 the above plus b_ lines would be retained 

d^« - 0.270 the above plus W lines would be retained 

d^. c' 0.400 all corners, lines and isolated pixels would be retained 

As seen in Figure 2, these predictions are accurate. The image was 

initiatilized very close to a fixed ooint by choosing the initial label 
estimates as p^*°(b or W) « 0.99, and thus could be regarded as an image 
v/hich has approached that condition by some preceding iterations of 
relaxation; moreover, it is useful to suppose the initial labeling represents 
the true labeling since then the label conversions observed in Figure 2 
would represent the introduction of labeling errors. 

An example with assumed initial labeling errors is shown in Figure 3. 

Again, initial label estimates of 0,99 were chosen to allow the prediction 
of (7) to be checked. As seen when d^. is chosen to avoid loss of corners, 
the relaxation process converges to the true labeling and remains there. 
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However, when Is less than the prescribed value, the corners are lost 
shortly after the true labeling has been achieved. 

The predictions of (5) through (9), of course, only hold exactly 
for an image that has approached a 0,1 fixed point and thus tacitly 
assumes that the local averaging that gives rise to the conversion of 
border labels takes place when the label estimates are all near 0 or 1. 

While this is indeed the case, averaging also takes place earlier when 
the label probability estimates are not quite so extreme. By initializing 
label probabilities further from a fixed point, the predictions from 
equations such as (7) through (9) will be modified. As an indication 
of this, Figure 4 illustrates how the value of d^ predicted from (7) 
for the example of Figure 3 is modified for a range of initial label 
estimates. This graph was produced empirically and implies that (7) 
is a lower bound. 

Should an exclusive neighborhood definition be used, then d^. » 0 
in (5) through (9). Thus Ai label deterioration of the types considered 
will occur unless the right-hand sides of those equations are less than 
zero. A little thought reveals that these equations can never be satisfied 
for all complementary pairs of neighbor geometry (i.e., Xi corners in Xz 
regions and Xz corners in Ai regions) so that label degradation leading to 
an increase in labeling error would always be expected to occur with 
conventional probabilistic relaxation algorithms applied to real imagery 
when used with exclusive neighborhood definitions. The supervised algorithm 
proposed recently, however, can be adjusted to avoid the degradation 
problem on exclusive neighborhoods since it bears similarity to using an 
inclusive neighborhood definition with the conventional algorithm [12]. 
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labeling improvement during relaxation 

The intention of applying relaxation to an image is to improve upon 
a labeling which has been generated beforehand by some "imperfect" process. 

In endeavoring to examine the improvement, it is useful to view the situation 
in the following manner. The relaxation algorithm does not know, of course, 
which are the correct and which are the incorrect labels. It only "knows" 
which labels are consistent and which are inconsistent with their neighbors. 
Consequently, an image with initial labeling errors will be treated by the 
relaxation algorithm as though it were correctly labeled and the "improve- 
ment" v/hich it creates is a conversion of locally inconsistent labels. 

This conversion will take place by mechanisms such as those described in 
the previous section and, in particular, for pixels that are close to 
fixed points, equations such as (7) through (9) can be used to describe 
labeling improvement in addition to likely degradation. Indeed, in the 
special case when an image is intentionally initialized close to a fixed 
point, those expressions can be used very accurately to describe the labeling 
improvement phase as well as any deterioration in the labeling that might 
occur. In such a situation, the predictions of (7) through (9) (for a 
two-label example) allow the value of d^. to be chosen relative to the 
compatibilities and other neighbor weighting coefficients to ensure that 
some labels are intentionally converted (i.e., those in error), while others 
are retained. Clearly the requirements for improvement and for avoiding 
degradation will often conflict in real image segments and, in order to 
obtain clean-up during relaxation, some correct labels may have to be 
sacrificed. As an illustration of these comments, consider the results 
of Figure 5 and suppose any one of the isolated pixels happened to be 
correctly labeled initially. If corner W labels are not to be lost, (7) 
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demands d^. ^ 0*274, whereas (8) and (9) require d^ < 0.36/', 0,429 
respectively if line end pixels and individual pixels have to be removed 
during the relaxation "improvement" phase. Choosing d. « 0.280 shorn that 
all erroneous labels are modified as expected, except that shown by the 
arrow which forms a corner with the corner W region. However, if one of 
the isolated pixels was correctly labeled initially, as supposed above, 
then this is also now in error. 

The discussions above and the supporting results presented have been 
based upon initial label probabilities being chosen close to 0 or 1. The 
graph of Figure 4 supports that these comments will apply also for initial 
label estimates different to 0, 1 but all being the same for the same 
class. Should the initial label estimates within a class be all different 
(as would happen, for example, should they be determined on the basis of 
Mahalanobis distance considerations in a classification [7]), some 
correctly (and weakly) labeled pixels will be removed preferentially during 
the early improvement phase in the relaxation. However, all label estimates 
will then move toward 0 or 1 and the remarks of Section 3 regarding deterioration 
still apply in principle. 

b. RELEVANCE OF ACCURATE COMPATIBILITIES 

In view of the comments of the previous two sections, it is clear that 
control of a relaxation process lies significantly in equations of the 
type (7) through (9) for a two-label problem and similar (albeit more 
numerous) manifestations of (5) for a multi-label exercise. Consequently, 
in the removal of initial labeling errors and in avoiding label degradation, 
the actual values of the compatibility coefficients (r. -(XlA') or p. .(X|X')) 

* J J J 

appear not to be important in pixel labeling so much as their values relative 


- 13 


to each other and to d^. as described in (7) through (9). As a demonstration 
of this, consider again the example of Figure 5, and arbitrarily choose the 
compatibilities as p^j(WlW) " 0,600 and p^.j (b|bj = 0.700 (compared with the 
true values of 0.500 and 0.875). Eqn (7) shows d^> > 0.091 for corner 
retention but loss of other geometries. Choosing d^. = 0.096, the results 
in Figure 6 are obtained showing the expected label improvement without 
subsequent degradation, notwithstanding the arbitrary choice of the 
compatibilities. 

DESIGN OF POSSIBLE RELAXATION STRATEGIES FOR PIXEL LABELING 
Since equations such as (7) through (9) describe the effect of relaxation 
(within the comments of the previous sections), it should be possible to 
specify an appropriate set of compatibilities and the weighting constant 
d^. to achieve certain desired results. In so doing, the following guide- 
lines are significant in a two-label situation, with the neighborhood 
definition chosen: 

(i) If p..(X?,|A?,) > p.,(XijXi) corner pixels labeled X?, protruding 

• J M 

into Xi regions will never degrade. 

(ii) Practical lower bounds on the compatibilities p.j(XilXi) and 

• J 

p 4 ^(X?.lXa) are 0.5. Otherwise the image must have consisted 
I J 

of isolated pixels or lines of pixels, depending upon the manner 
in which those compatibilities were calculated. 

(iii) for both p^-^-(XijXi) and p- -(X?. [Xz) dose to 0.5 (7) through 
(9) reveal that all conditions on d^. are approximately zero — 
i.e., there would be no degradation and no improvement (as 
expected). 


14 


(iv) Convergence is faster for larger values of the compatibilities. 

As a result of the above comments, the following is proposed as one possible 
design strategy for pixel labeling relaxation procedures, which obviates 
the need to obtain reliable estimates for the compatibility coefficients. 

It turns out to be a sub-optimal procedure for practical image data, since 
its success depends upon forekno'/le'j.^e of, or a feeling for, the prevailing 
geometry in a particular image; however, it is a significant improvement 
on choosing neighbor weights (d.) in an arbitrary manner. 

J ‘ 

1. Choose all initial label estimates as 0.99 (or 0.01 as 
appropriate), unless there is good reason for doing other- 
wise. Such a choice (close to a fixed point) allows moderate 
accuracies in predictions made from (7) through (9). 

2. Choose the compatibility for the most prevalent class to 

be the strongest since this automatically preserves corners 
in that class and will preempt lower overall final error. 

3. Form an impression of the label geometries in which most label 
errors seem to lie (such as isolated pixels) and also of the 
label geometries which should not be allowed to degrade (such 
as corners) and choose d^. in order to remove only suspected 
errors. 

4. If speed is a consideration, choose the magnitudes of the 
compatiabilities to be as large as possible within the restraints 
imposed by the above considerations. 

This procedure is now illustrated using two data sets. One consists 
of a multi temporal Landsat image acquired over a region in Kansas and 
contains an array of 117 x 196 pixels. The other is a 40 x 100 pixel 
portion of that same image. The latter was chosen to enable the results 
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to be inspected on a pixel-by-pixel basis, whereas the former is used 
to illustrate performance on imacjery of the size typically encountered in 
remote sensing applications. Figure 7a represents two-category (wheat 
and non-wheat) ground truth for the smaller image and as such can be 
regarded as true labeling. Figure 7b shows the result of a crude 
classification of that portion. This classification was obtained using 
a minimum distance to means classifier on pattern vectors consisting of 
three only of the 16 possible spectral response features. These features 
were chosen beforehand on the basis of a separability measure computed 
over the full image. Similarly, training areas were selected from the 
full segment and Figure 7b represents only a small portion of the 
resulting classification map. 

Inspection of Figure 7a suggests that it would be desirable to 
remove isolated W (wheat) pixels from any classification but that W line 
ends and corners of W fields should not be allowed to alter. Also, it 
would seem desirable to preserve blank (non-wheat) corners. On the basis 
of these observations, the relaxation parameters 

p(W|W) = 0.600 p(b|b) = 0.700 d. = 0.200 
could be suggested as a possible choice which would remove scattered W 
pixels but retain all other geometries. This prediction can be checked 
on Figure 7c which shows labeling error, after 100 iterations of relaxation, 
as a function of d^. ; the image was initialized close to a fixed point. It 
is evident that d^. = 0.200 is a good choice for these particular data. 

The figure also displays discernible improvements in labeling at values 
of d^. corresponding to the preservation of the various geometric features 
noted on the diagram. Inspection of Figure 7d, which shows the final 
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labeling achieved with * 0.200, reveals that only the isolated W 
pixels have been relabeled as required. 

In passing, it is of interest to note that only when d^. > 0.15, 
relaxation reduces the labeling error below that in the initial labeling. 
For d^. < 0.14, the label degradation mechanism leads to worse error after 
relaxation than before for these particular data. This effect is so severe 
here because of the "large" number of geometries that need to be preserved. 

As a basis for comparison of the results of Figure 7, Figure 8 shows 
label error as a function of d^. using "true" compatibilities calculated 
from the full image. Shown in that figure also are the predicted values 
of d^. from (7) through (9) relating to the preservation of particular 
geometries. Comparison of Figures 7c and 8 shows no essential difference 
in shape, supporting the notion that exact compatibilities are not 
required. 

Figure 9 shows the result of relaxation over the complete image using 
the "true" values of the compatibilities. Again, the significant values 
of d^. are noted. Examination of this figure reveals that preservation of 
(too many) wheat corners is detrimental. The fact that this behavior is 
different from that of Figures 7 and 8 is indicative of the fact that the 
portion of the image used in these previous figures has a geometric 
character that is not representative of the complete segment. This is 
evident from an inspection of the full ground- truth map. 

Figure 10 shows labeling error versus number of iterations for 
selected values of d^ in Figure 9. Note that for d^ less than optimum, 
labeling error initially decreases, passes through a turning point, and 
increases again before settling down to a pessimistic final value. For 
values of d^. near 0.15, the error curve does not exhibit the deterioration 
phase and has a final value which is almost as low as the minimum in the 
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previous curve. For larger d^, while the curve is monitonically decreasing 
the final error is larger than necessary. Ultimately, for large d^ the 
curve will remain constant at the initial labeling error. 

7. S UPERVISION AS A MEANS FOR SEGMENTING LABEL CONVERSION EFFECTS 

An unfortunate observation on the results of the previous section is 
that once a choice of algorithm parameters has been made to retain some 
and remove other border geometries, this effect takes place over the 
complete image, except at those pixels that were so strongly labeled 
initially that their probability estimates reach 0 or 1 well ahead of 
others. In a practical pixel labeling exercise, particularly of the remote 
sensing variety, this is undesirable since there could be segments of an 
image where, for example, single pixels would wish to be preserved, 
whereas in other segments single pixels would want to be removed (e.g., 
urban versus agricultural regions). The only v</ay this effect can be 
implemented with existing algorithms is to attempt to condition the 
initial probability estimates by, say, strengthening those corresponding 
to regions where single pixels are desirable. At best, this would be a 
time-consuming procedure that would also override any information implicit 
in the initial labeling. An alternative, and potentially attractive, 
technique for suitably segmenting the image is to make use of the super- 
vised approach to relaxation proposed recently [12]. In that technique, 
the label estimates at each iteration of relaxation are modified by 
reference to some other data. Using the initial label estimates for 
that other data has given rise to a relaxation procedure that can be used 
to overcome the detrimental label conversion effects on boundaries, as 
discussed earlier. However, it should also be possible to derive the 
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supervising data from an array which overlays the image and which contains 
data on the likely geometries which should desirably be preserved in 
various image segments. As an early indication of the viability of such 
a scheme, Figure 11 shows an image which is supervised with the overlay 
array of probabilities indicated. As observed, it is possible to adjust 
the degree of influence of the supervision (via the parameter 3 ) to retain 
certain geometric features in one image segment while allowing those same 
features to be relaxed out in other segments. Means for establishing the 
appropriate values of 3 have not yet been determined; hov/ever, it is 
believed that the results of Figure 11 demonstrate the usefulness of this 
approach. 

8. DISCUSSION AND CONCLUSIONS 

From the results presented in the previous sections, it is evident 
that the compatibilities should not need to be accurately characteristic 
of a particular image. Rather, as noted, it is biases in the compatibilities, 
along with the value of d^. (relative to the weights on the other neighbors — 
here all taken to be the same) that substantially determine how relaxation 
will behave on particular image data. This is demonstrated in the fact 
that the compatibilities in Figures 5 and 7 are the same (by choice) and 
yet clearly the images are quite different. In those cases it was only 
necessary to choose the appropriate value for d^. in view of the compatibilities 
given. A little thought also reveals that for image data (of the Landsat 
type especially) the true compatibilities cannot be particularly significant 
since these are statistically averaged measures computed over the whole 
or even a part of an image where in fact some regions of an image may 
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bear no geometric or stati 5 tical resemblance to other areas of that 
same image. 

The examples presented above have shown that it is possible with 
image data to choose compatibilities and specific values of the neighbor 
weights d. such that the relaxation process will converge to a near optimum 

J 

error which will not subsequently increase owing to label conversion 
(degradation) mechanisms. Owing to the greater degree of homogeneity 
present in the data typically encountered in picture processing tasks 
(such as noise removal), it is likely that the predictions concernirjg 
retention of label geometries presented herein may be more useful in 
those applications than with pixel labeling. For example, inspection of 
the noisy scene of a house used in [7] reveals that the most important 
features to be retained are the corners of one label type within another. 
For example, it is important to preserve corners of "sky" which protrude 
into regions of "brick" (sky is visible through the end of a veranda). 

With the compatibilities chosen by those authors, (5) can be used to 
specify a value of d^. beyond which sky- in-brick corners will be preserved. 
Those authors use a full 3x3 neighborhood and choose d- = l,Vj (the 

J 

compatibility coefficients were defined by mutual information). Choosing 
dj = 1,V jYI , eqn (5) shows that for sky-in-brick corners to be preserved, 

J 

it is necessary that d^. ^ 5.22 (the actual value of d^ depends upon the 
initial label estimates). Clearly with d^. = 1 degradation will occur and 
it is to be expected, with regard to this feature at least, that label 
accuracy would improve early on during relaxation and then become poorer 
owing to sky label degradation. Indeed those authors report a degradation 
phase during this relaxation exercise and it is probably that a (major) 
component of it is a result of sky label loss at corners. 


The theme of this paper has been to develop a rndel for the 
relaxation process that would permit the label degradation mechanism to 
be understood and thus avoided. Consequently attention has not been given 
to attempting to achieve the smallest possible labeling error. For example, 
the exercises presented have been initialized with label probabilities 
all close to a fixed point and all the same within the same class initially. 
Distributing the initial label estimates, however, according to some 
measure of confidence, would probably lead to overall lower error since 
erroneous labels that were weak initially would be removed before the 
relaxation mechanism fixed them by one of the preservation measures discussed 
earlier. Notwithstanding this, the predictions of equations such as (5) 
through (9) are important guidelines for controlling the relaxation 
improvement and degradation mechanisms and consequently for algorithm design 
as discussed. 
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Figure 1. Neighborhood definition used herein. 



Figure 4. Modification of the orediction of equation '7) for 
label estimates further from a 0,1 fixed point. 
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Figure 2. Verification of predictions made using equations 
(7) through (9). 
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Figure 5. Illustration of a choice of the neighbor weight (d.), 
associated with the central pixel of the neighbor-^ 
hood, to remove individual pixels and line-ends but 
to preserve corners. If an individual pixel was 
correctly labeled, as indicated, then it becomes 
erroneous. 
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Figure 6. Repetition of the example of Figure 5 but with 
"arbitrarily" chosen compatibilities and with d 
adjusted accordingly. 
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ORIGINAL PAGl'J IS 
OF POOR QUALITY 

Figure 7(a), Two-category (wheat-Wj nonwheat-blank) ground 

truth for the 40x100 pixel Landsat multi temp oral 
small image. This is considered as "true" labeling. 
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Figure 7 (c). Remaining labeling error as a function of the 

central pixel neighbor weight d. , after 100 iterations 
of relaxation on the 40x100 pixel small image. 

(100 iterations is sufficient to achieve the final 
labeling.) The compatibilities have been chosen 
as discussed in the text. 
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Fiqure 7(d). Final labeling of the small image after 100 
iterations with d. = 0.200. Note that only 
isolated W pixels^ have been removed but that 
all other geometric features both in the W 
and blank labels have been retained. 
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Figure 8. Effect of relaxation on the small image when the 
"true" values of the compatibilities are used. 
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pC bib) *0.971 
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p°Cw) = p'’Cb)*0.99 


Figure 9. Remaining labeling error after 100 iterations of 

relaxation on the full 117x196 pixel Landsat image, 
using the "true" compatibilities. 



Labeling error versus number of iterations when 
relaxation is applied to the 117x196 pixel image, 
using several values of d^ from Figure 9. 


Figure 10. 
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Figure 11. Illustration of the usefulness of supervising the 
relaxation process with a map of probabilities in 
order to segment the label conversion process. The 
image was initialized with p°(W or = 0.99 and 
300 iterations of relaxation were applied. 



